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Ada  is  already  required  for  use  in 
embedded  sohware  of  the 
United  States  Department  of  Defense. 
In  time,  more  government  agencies 
and  private-seaor  companies  will  be 
using  Ada,  yet  it  is  not  without  its 
problems.  Though  praised  for  being 
comprehensive,  Ada  has  also  been 
criticized  for  being  cumbersome.  Its 
effective  use  by  people  with  varied 
backgrounds  will  depend  on  training, 
available  tools,  and  the  application  of 
both  to  the  project  at  hand. 

This  article  examines  the  use  of  Ada 
in  a  software  projea  developed  by  the 
General  Electric  Company.  The  proj¬ 
ect  was  monitored  by  the  University  of 
Maryland  and  GE  to  identify  areas  of 
success  and  difficulty  in  learning  and 
using  Ada  as  both  a  design  and  a 
coding  language.  Since  production- 
quality  Ada  translators  were  not  readi¬ 
ly  available,  the  study  focused  on 
training  and  early  software  devel¬ 
opment  .v^J/c_^us  on  the  use  and  ef¬ 
fect  of  AdaotTtWrptQjgcri  which  was 
conducted  primarily  inT9S2;> Our 
study  also  presents  the  major  factors 
to  consider  before  u.sing  Ada  in  soft¬ 
ware  development,  particularly  when 
training  in  Ada  is  necessary r-AIthotigh 
many  of  our  conclusions  may  seem  ob¬ 
vious  now,  they  were  unexpected  when 
(his  project  began. 


/^Our  study  attempts  to  meet  several 
goals.  The  first  focuses  on  characteri¬ 
zation  of  the  effort,  the  changes,  and 
the  errors  of  the  project.  The  second 
considers  how  Ada  was  used  on  the 
project.  The  third  concerns  evaluation 
of  the  data  colleaion  and  validation 
process,  while  the  fourth  concentrates 
on  the  development  of  measures  for 
the  Ada  Programming  Support  Envi¬ 
ronment.  ^ _ _ — — 

The  development  of  metrics  is  an 
on-going  projea  discussed  by  Basili 
and  Katz'  and  Gannon  et  al.,^  so 
Ada-specific  metrics  will  not  be  dis¬ 
cussed  here  except  as  they  apply  to  this 
project. 

Data  sources  and  validation 

From  the  start  of  the  project,  data 
was  collected  from  a  variety  of 
sources.  Change  data,  in  particular,  is 
often  gathered  only  after  the  code  has 
been  compiled  and  entered  into  a 
"system”  for  use  by  other  project 
members.  In  this  project,  however,  all 
changes  made  after  the  text  was  on-line 
were  included.  Therefore,  compari¬ 
sons  with  data  of  other  projects  may 
be  misleading.  However,  some  of  the 
data  will  be  presented  with  compiler- 
detectable  faults  eliminated  so  com¬ 
parisons  can  be  made  with  the  early 
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Figure  1.  Change  request  form. 

stages  of  other  early  Ada  projects.  Our 
data  is,  however,  incomplete  because 
the  project  is  incomplete;  however,  it 
might  be  useful  for  comparison  to 
other  projects. 

Data  sources.  The  programmers 
were  asked  to  complete  various  forms. 
Each  week,  a  component  status  report 
form  indicated  how  each  programmer’s 
effort  was  distributed  by  component 
involved  and  by  phase  of  devel¬ 
opment.  Each  time  a  need  for  change 
arose,  a  change-request  form  was 
completed.  If  the  change  was  an  error 


correction,  an  error  description  form 
was  also  used.  For  every  component 
involved  in  a  change,  an  individual 
document  change  repon  form  was 
filled  out  indicating  that  the  librarian 
made  the  change  and  documenting  the 
time  needed  to  put  the  change  into  the 
code  or  design.  Copies  of  the  change 
report  form  and  the  error  report  form 
can  be  found  in  Figures  1  and  2. 
Copies  of  the  other  forms  can  be 
found  in  Basili  ct  al. ' 

In  addition,  a  copy  ol  c.ach  design 
and  code  versi^in  of  evt-ry  module  vs  as 
kept.  The  source  code  measures  were 


taken  from  the  latest  version  for  each 
module. 

A  simple  parser  for  Ada  and  the 
PDL  was  written  to  gather  the  various 
code  measures  and  to  check  the  syntax 
of  the  PDL.  It  will  be  enhanced  to 
gather  more  complicated  measures  as 
we  understand  more  about  Ada. 

Data  validation.  Data  collected  on 
forms  is  difficult  to  validate,'*  but 
valid  data  is  necessary  for  valid  results; 
therefore,  the  completed  forms  were 
screened  for  inconsistencies.  The  pro¬ 
grammers  could  then  clarify  discrep¬ 
ancies  before  data  was  analyzed. 
Screening  should  be  done  as  soon  as 
possible  in  future  studies.  If  the  forms 
could  be  completed  on-line,  checked 
for  inconsistencies,  and  automatically 
entered  into  a  database,  validation 
could  proceed  more  quickly.  We  sug¬ 
gest  that  future  data  collectors  create  a 
checklist  of  their  expectations  at  each 
milestone  of  the  project. 


The  project 

This  study  was  arranged  to  monitor 
training,  designing,  coding,  and  unit 
and  system  testing  of  a  realistic  in¬ 
dustrial  software  development  proj¬ 
ect.  We  initially  planned  to  study  the 
entire  software  development,  but  with¬ 
out  a  production-quality  Ada  com¬ 
piler,  the  task  was  nearly  impossible. 
However,  many  of  our  observations 
about  early  development  may  apply  to 
other  projects  using  Ada. 

We  emphasize  that  this  project 
began  in  January  1982,  when  many  ex¬ 
pected  that  a  couple  of  weeks  would  be 
surficient  for  programmers  to  learn 
Ada.  Many  of  those  expectations  have 
changed  now  that  we,  and  others,  have 
used  Ada.  At  least  some  of  these 
changes  were  caused  by  early  observa¬ 
tions  of  and  discussions  about  this 
project.  However,  this  study  provides 
some  empirical  evidence  lor  certain 
now  widely-held  observations. 
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Programniers  and  training.  Four 
programmers  with  diverse  back¬ 
grounds  were  selected  in  order  to 
examine  whether  a  programmer’s  ex¬ 
perience  and  education  would  in¬ 
fluence  his  understanding  and  use  of 
Ada.  Table  1  shows  the  education,  ex¬ 
perience,  and  language  knowledge  of 
each  programmer,  but  a  sample  of 
four  programmers  can  only  hint  at 
possible  influences.  A  more  detailed 
study  of  background  and  performance 
is  presented  by  Bailey.^ 

None  of  the  programmers  knew 
Ada  before  the  project  began;  they 
volunteered  to  learn  Ada.  Therefore, 
they  were  probably  more  enthusiastic 
than  most  programmers  about  using 
the  language.  Since  this  was  an  early 
project,  the  training  was  longer  (one 
month)  and  more  comprehensive  than 
the  industry  standard  at  the  time. 

The  training  began  with  15  hours  of 
videotaped  lectures  by  Ichbiah,  Firth, 
and  Barnes  over  a  period  of  four  days. 
Six  days  of  in-house  training  by 
George  W.  Cherry  in  Ada  syntax  and 
basic  concepts  spread  over  the  follow¬ 
ing  four  weeks.  During  this  time,  the 
programmers  also  practiced  writing 
Ada  programs,  read  the  Ada  reference 
manual,  and  reviewed  their  class 
notes.  The  NYU  Ada/Ed  interpreter 
was  used  for  programming  assign¬ 
ments,  which  included  a  500-line  team 
project. 

However,  as  usual,  the  program¬ 
mers  had  little  experience  with  many  of 
the  software  engineering  practices  that 
Ada  was  designed  to  support.^  They 
did  have  varying  degrees  of  experience 
with  structured  software  development 
practices,  e.g.,  design  and  code  walk¬ 
throughs,  structured  programming, 
and  program  design  language.  They 
were  given  a  half-day  review  of  soft¬ 
ware  development  practices  by  Victor 
R.  Basili  to  provide  them  with  a  com¬ 
mon  perspective  oti  these  techniques. 
At  the  time,  such  trainine  in  mcihod- 
oloe\  was  considered  sull'icient;  how¬ 
ever,  further  discussions  on  meilu'd- 
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Figure  2.  Error  report  form. 


September  1985 


Table  2.  Size  eharactaristica  of  ttM  product. 
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NON-BLANK  LINES 
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ology,  especially  the  use  of  abstractions 
during  design,  occurred  during  the 
projea  as  the  programmers  used  the 
techniques. 

This  approach  (Ada  first,  software 
engineering  techniques  later)  did  not 
seem  to  give  the  programmers  the 
appropriate  model  for  learning  how  to 
use  Ada  to  support  the  software  engi¬ 
neering  concepu.  They  had  only  the 
programming  model  from  their  pre¬ 
vious  language  experience,  predomi¬ 
nantly  Fortran.  Prior  training  in  the 
software  engineering  concepts  might 
have  better  prepared  them  for  learning 
Ada.  Bailey  attempted  to  compare  the 
two  training  approaches  (Ada  first  and 
software  engineering  concepts  second 
versus  the  opposite  ordering).^  His 
study  tried  to  correlate  the  program¬ 
mers’  background  and  the  order  of 
concept  presentation  with  success  in 
the  classes  he  studied.  However,  the 
results  were  inconclusive  and  indicated 
that  more  studies  arc  needed  to  deter¬ 
mine  how  Ada  should  be  taught. 


DevelopmenI  and  product.  The 
project  under  study  involved  the  re¬ 
design  and  implementation  in  Ada  of  a 
portion  of  a  satellite  ground  control 
system  originally  written  in  Fortran.  It 
included  an  interactive  operator  inter¬ 
face,  graphic  output  routines,  and 
concurrent  telemetry  monitoring.  The 
programmers  never  saw  the  compar¬ 
able  Fortran  source  programs.  Be¬ 
cause  pieces  of  the  subset  were  scat¬ 
tered  throughout  the  original  system 
and  the  developed  system  contained 
some  added  features,  determining  the 
precise  size  of  the  subset  as  imple¬ 
mented  in  Fortran  was  difficult.  How¬ 
ever,  the  subset  was  estimated  to  con¬ 
tain  between  5000  and  8000  text  lines 
of  Fortran,  including  declarations  but 
not  comments  or  blank  lines. 

The  project  began  in  February  1 982 
and  ended  in  July  1983.  However, 
most  of  the  development  took  place 
between  February  and  December 
1982.  Some  testing  was  done  between 
May  and  July  1983.  Requirements 


analysis  was  done  prior  to  and  concur¬ 
rently  with  training,  and  an  Ada-like 
Program  Design  Language  was  used 
to  design  the  system. 

The  PDL  has  two  levels.  The  first 
describes  the  input,  output,  and  excep¬ 
tions  for  the  module,  includes  a  brief 
abstract  of  what  the  module  should 
do,  and  outlines  the  algorithm.  The 
entire  first  level  is  written  in  .Ada 
comments. 

The  second  level  is  a  more  detailed 
description  of  the  algorithm  with  a 
combination  of  Ada  and  PDL  escapes 
enclosed  in  braces.  These  escapes  can 
replace  any  Ada  nonterminal,  though 
they  usually  replace  statements  and 
conditionals.  The  first  level  remains  as 
documentation  for  the  code. 

After  design,  the  system  was  coded 
in  Ada,  and  some  of  it  was  unit  tested. 
As  the  projea  was  begun  before  pro- 
duaion-quality  Ada  compilers  were 
available,  it  was  not  complaely  coded 
or  tested.  About  750  lines  of  PDL  text 
were  left  uncoded.  Some  unit  testing 
was  done  with  the  NYU  Ada/Ed  inter- 
preter  and  the  ROLM  compiler,  but 
no  system  testing  was  conducted. 

The  product  was  examined  by  a 
number  of  people  interested  in  but  not 
associated  with  the  projea,  such  as  the 
developers  of  the  original  project.  The 
design  was  judged  to  be  funaional 
rather  than  object-oriented.  This  as¬ 
sessment  was  not  surprising  since  the 
programmers  were  most  familiar  with 
Fortran  and  its  funaional  approach 
and  the  requirements  were  funaional 
in  nature.  In  faa,  the  high-level  Ada 
design  was  very  similar  to  the  original 
Fortran  design. 

In  order  to  provide  an  initial 
charaaerization  of  this  projea.  Table 
2  provides  some  size  data  for  the 
design  and  code.  Note  that  some 
design  was  never  expanded  into  code 
because  the  project  was  not  com¬ 
pleted.  In  addition,  many  sections  of 
code  were  copied  almost  verbatim 
from  the  corresponding  PDL.  All  but 
four  of  the  modules  with  fKMh  PDL 
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and  code  had  a  text  expansion  ratio 
below  two  to  one.  Of  the  remaining 
four  modules,  three  had  expansion 
ratios  just  over  two  to  one,  and  one 
module  expanded  from  29  text  lines  to 
124.  Therefore,  the  total  section  in¬ 
cludes  only  code  and  non-expanded 
design.  Nonblank  lines  of  source  in¬ 
clude  comment  lines  but  not  blank 
lines.  Text  lines  must  have  some  Ada 
or  PDL  on  them.  Executable  state¬ 
ments  do  not  include  declarations 
unless  there  is  an  initialization  in  the 
declaration. 


Factors  affecting  the  data 

Several  factors  affected  the  out¬ 
come  of  this  study,  and  understanding 
them  is  important  for  proper  inter¬ 
pretation  of  the  results.  Many  will  not 
be  present  in  later  Ada  developments, 
but  the  training  and  tool  issues  that 
clearly  affected  this  project  will  affect 
others  as  Ada  use  increases. 

The  useful,  but  very  slow,  NYU 
Ada/Ed  interpreter  became  unusable 
toward  the  end  of  the  project,  as  the 
size  of  the  developing  system  grew. 
This  difficulty  had  a  demoralizing  ef¬ 
fect  on  the  programmers,  and  they  did 
not  finish  coding  or  testing  the  projea. 
When  the  ROLM  compiler  became 
available,  further  testing  was  done. 

The  results  set  forth  are  based  on 
data  collected  through  coding  and 
some  unit  testing.  In  addition,  the  vast 
majority  of  the  Ada-related  errors 
either  were  or  could  have  been  detected 
by  a  compiler.  The  dominance  of  these 
errors  might  have  diminished  had  the 
code  been  executed  and  testing  com¬ 
pleted.  Many  more  logic  errors  might 
have  been  uncovered  had  all  the 
modules  undergone  error-free  com¬ 
pilation. 

Many  trivial  errors  that  might  have 
been  detected  by  a  PDL  processor  ap¬ 
peared  in  the  design.  Some  of  these 
were  detected  during  design  readings 
and  rev  iews  and  were  removed.  Others 


Table  3.  Effort  for  each  phase  of  the  project. 


PROJECT  PHASE 

AMOUNT  OF  TIME 
(IN  HOURS) 

PERCENTAGE 

REQUIREMENTS  ANALYSIS 

530.S 

12.73 

REQUIREMENTS  WRITING 

113.6 

2.73 

DESIGN  CREATION 

514.4 

12.34 

DESIGN  READING 

37.7 

0.91 

FORMAL  DESIGN  REVIEW 

162.4 

3.89 

CODING 

305.6 

7.33 

CODE  READING 

13.3 

0.32 

FORMAL  COOING  REVIEW 

62.3 

1.50 

UNIT  TESTING 

332.7 

7.98 

INTEGRATION  TESTING 

0 

0.00 

REVIEW  TESTING 

0 

0.00 

TRAINING 

849.1 

20.38 

OTHER  ACTIVITY 

1245.7 

29.89 

TOTAL  REQUIREMENTS 

644.1 

15.46 

TOTAL  DESIGN 

714.5 

17.14 

TOTAL  CODE  DEVELOPMENT 

381.2 

9.15 

TOTAL  TESTING 

332.7 

7.98 

TOTAL  TRAINING 

849.1 

20.38 

TOTAL  OTHER  ACTIVITY 

1245.7 

29.89 

ENTIRE  PROJECT 

4167.3 

100.00 

remained  until  the  code  developed  Effort 
from  the  design  was  compiled.  Many 

of  these  later  changes  were  not  made  in  The  first  goal'of  the  study  is  to 
the  design;  therefore,  seme  of  the  characterize  the  effort  expended  on  the 
design  and  code  documents  were  in-  project.  By  doing  so,  we  can  provide 
consistent.  In  addition,  design  reviews  insight  into  how  programmer  time 
tended  to  focus  on  the  numerous,  easi-  might  be  used  in  future  Ada  projeas 
ly  detected,  trivial  errors  rather  than  and  a  basis  for  comparison  with  later 
on  the  deeper  design  issues  and,  per-  Ada  projects, 
haps,  errors.  A  PDL  processor  would  Table  3  shows  the  time  spent  on 
have  changed  this  focus.  each  phase  of  the  project,  including 

Type  and  quantity  of  training  were  training.  Productivity  was  calculated 
other  factors.  Twenty  percent  of  the  from  the  total  lines  in  Table  2  and  the 

total  effort  was  spent  on  training,  total  design  and  code  development 

Software  engineering  concepts  such  as  time  in  Table  1 .  For  each  hour  spent  in 
data  abstraction  and  information  design  and  code  development,  9.03 
hiding  were  not  stressed  during  Ada  nonblank  lines  of  code  and  4.70  text 
training,  although  they  were  presented  lines  were  developed.  The  values  are 
to  the  programmers  afterward.  The  upper  bounds  (and  may  not  be  mean- 
programmers  indicated  that  training  ingful  since  the  project  was  not  com- 
was  insufficient,  and  their  use  of  Ada  pleted). 
suggests  that  they  probably  needed 
more.  Therefore,  we  must  conclude 
that  a  sizable  effort  will  be  needed  to  Changes 
learn  .Ada  and  must  be  considered 

w  hen  planning  carls  projects  using  the  Our  second  goal  is  to  characterize 
language.  the  changes  in  the  project  in  order  to 
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Table  4.  Breakdown  of  changes  by  type. 


NUMBER 

TYPE  OF  CHANGE _  OF  CHANGES _ PERCENTAGE 

ERROR  CORRECTIONS  192  56.96 

CHANGES  IN  PROBLEM  DOMAIN  1  0.29 

PLANNED  ENHANCEMENTS  9  2.67 

AVOIDANCES  OF  APPARENT  PROBLEMS 

WITH  THE  COMPILER  18  5.37 

AVOIDANCES  OF  OTHER  PROBLEMS  IN 

THE  DEVELOPING  ENVIRONMENT  2  0.59 

ADAPTATIONS  TO  A  CHANGE  IN  THE 

DEVELOPING  ENVIRONMENT  7  2.08 

IMPROVEMENTS  OF  DOCUMENTATION, 

CLARITY,  OR  MAINTAINABILITY  76  22.55 

OPTIMIZATION  OF  TIME,  SPACE,  OR 

ACCURACY  2  0.59 

INSERTION  OR  DELETION  OF  DEBUG 

CODE  9  2.67 

OTHER  THAN  ABOVE  21  6.23 


determine  how  the  product  evolves,  show  that  they  were  concerned  about 
The  classification  of  changes  can  in-  clarity  and  documentation, 
dicate  which  factors  might  have  af-  The  time  to  determine  the  need  for 
fected  the  projea.  Information  on  how  change  was  one  hour  or  less  in  almost 
easily  the  product  was  changed  might  all  cases.  In  addition.  46  percent  re¬ 
indicate  the  quality  of  the  product.  quired  only  six  minutes.  The  need  for 
Analysis  of  the  337  change  request  these  changes  was  easily  determined, 
forms  (Figure  1 )  and  the  439  individual  Few  changes  took  much  longer  than  a 
document  change  forms  indicates  that  half  hour,  although  four  changes  re- 
the  effect  of  Ada  on  the  changes  made  quired  more  than  one  day  to  determine 
m  the  project  cannot  be  distinguished  they  were  needed:  two  were  planned 
from  the  effect  of  any  other  factor,  enhancements:  one  was  an  avoidance 
Code  changes  accounted  for  61  per-  of  a  problem  with  the  compiler;  and 
cent.  As  stated  previously,  however,  the  last  involved  the  creation  of  a 
many  of  these  changes  were  errors  global  definitions  package  that  inter- 
which  should  have  been  caught  at  the  faced  with  several  components, 
design  stage.  Thirty-two  percent  of  the  The  amount  of  time  needed  to 
changes  were  in  design  documents,  design  and  implement  changes  was 
and  only  seven  percent  were  in  re-  also  minimal.  The  majority  took  one 
quirements  documents.  hour  or  less.  Of  the  code  changes,  all 

The  breakdown  by  type  of  change  is  but  five  took  two  hours  or  less.  Two 
shown  in  Table  4.  The  majority  (57  changes,  which  took  three  hours  and 
percent)  of  the  changes  were  error  cor-  one  day,  respectively,  involved  avoid- 
rections  which  will  be  described  in  ing  problems  with  the  compiler.  One 
detail  later.  Of  the  non-error  changes,  change,  which  took  one  and  a  half 
52  percent  were  improvements  of  clari-  days,  was  an  adaption  to  a  change  in 
ty,  maintainability,  and  documenta-  the  development  environment.  One 
tion.  The  low  number  of  planned  code  change,  which  took  four  hours, 
enhancements  indicates  that  the  pro-  was  an  error  correction  and  will  be 
grammers  tried  to  implement  portions  discussed  in  the  errors  section.  The 
of  the  system  immediately  rather  than  global  definitions  package  was  imple- 
start  with  a  subset  and  enhance  it  later,  mented  in  four  days.  The  .'ew  other 
The  large  numlicr  of  improvements  changes  which  took  much  longer  than 


usual  were  mostly  planned  enhance¬ 
ments  and  improvements  of  clarity, 
maintainability,  and  documentation 
of  requirements  documents.  A  change 
that  took  one  week  was  a  planned  en¬ 
hancement  in  a  requirements  section. 

The  total  time  spent  determining  the 
need  for  changes  then  implementing 
them  was  426.4  hours,  10  percent  of 
the  total  effort  for  the  entire  project. 
The  average  cost  was  1 .27  hours  per 
change,  but  more  than  80  percent  of 
the  changes  took  much  less  time. 

Components  involved.  We  deter¬ 
mined  the  number  of  components 
altered  in  each  change.  Seventy-seven 
percent  of  the  changes  caused  only  one 
component  to  be  modified,  but  up  to 
five  components  were  modified  in 
some  changes.  We  also  identified  70 
interface  changes  (21  percent  of  all 
changes)  defined  as  those  that  entail  a 
change  in  more  than  one  component  at 
the  same  level  of  document.  Only  2.9 
percent  of  these  were  in  the  require¬ 
ments;  the  rest  were  equally  divided 
between  design  antTcpde.  As  many  as 
five  components  were  altered  in  these 
interface  changes. 

From  this  data,  we  conclude  that 
most  of  the  changes  were  trivial  and 
involved  a  single  component.  Ada 
seemed  to  have  little  effect  on  the  non¬ 
error  changes.  Most  of  the  changes 
were  error  corrections,  but  many  were 
improvements  of  documentation,  clari¬ 
ty,  or  maintainability.  We  do  not 
know  how  this  distribution  would 
change  if  more  testing  were  done;  but, 
we  strongly  suspect  that  the  number  of 
error  corrections  would  increase. 

Errors 

Since  Ada  is  a  new  language,  pro¬ 
grammers  will  make  some  errors  w  hen 
using  its  new  features.  By  determining 
types  of  errors  made,  we  can  focus 
training,  tools,  and  techniques  on 
eliminating  or  detecting  the  mo'^i 
prevalent  or  severe  errors. 
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We  examined  192  error  description 
forms  (Figure  2).  Each  corresponds  to 
a  change  request  that  falls  into  the  er¬ 
ror  correction  category.  We  used 
several  different  error  classification 
schemes  to  understand  which  errors 
occur  and  how  to  detect  or  prevent 
them.  Note  that  our  figures  (Table  S) 
differ  slightly  from  Basil!  and  Per- 
ricone^  because  some  classifications 
were  changed,  and  the  data  were  inter¬ 
preted  in  light  of  these  changes. 

We  used  the  definitions  of  errors, 
faults,  and  failures  of  the  IEEE  Glos¬ 
sary  of  Software  Engineering.’  A 
“fault”  is  a  specific  manifestation  in 
the  source  code  of  a  programmer  “er¬ 
ror.”  A  single  “error”  can  result  in 
many  “faults.”  A  “fault”  may  cause  a 
“failure”  when  the  program  is  exe¬ 
cuted.  Errors  were  reported  for  this 
project,  but  few,  if  any,  failures  were 
reported  because  little  testing  was 
done. 

Document  type.  A  common  classifi¬ 
cation  of  errors  is  by  type  of  document 
and  how  it  was  involved.  If  we  know 
the  documents  involved,  we  can  ex¬ 
amine  them  more  closely  for  faults  or 
concentrate  on  their  careful  devel¬ 
opment.  Table  5  shows  a  breakdown 
of  the  enors  by  document  type.  We 
can  see  that  the  majority  (79  percent) 
of  errors  were  due  to  incorrect  code. 
Most  of  the  remaining  errors  were  at¬ 
tributable  to  incorrect  design.  Few  er¬ 
rors  involved  those  requirements, 
probably  because  those  requirements 
had  already  been  used  on  a  previous 
projea  and  were  fairly  well  written. 


Detection  and  correction.  If  we 
knew  which  activities  were  most  often 
successful  at  detecting  errors,  we  could 
concentrate  training  and  tool  devel¬ 
opment  to  support  them.  In  this  proj¬ 
ect,  compilation,  design  reading, 
design  walkthroughs,  and  code  read¬ 
ing  were  most  often  used  to  detect  er¬ 
rors.  Approximately  half  of  the  errors 


Table  5.  Errors  by  type  of  document. 


TYPE OF  DOCUMENT 

AND  HOW  INVOLVED 

NUMBER 

OF  ERRORS 

PERCENTAGE 

REQUIREMENTS  INCORRECT 

2 

1.04 

REQUIREMENTS  MISINTERPRETED 

4 

2.08 

DESIGN  INCORRECT 

29 

15.10 

DESIGN  MISINTERPRETED 

0 

0.00 

CODE  INCORRECT 

151 

78.65 

EXTERNAL  ENVIRONMENT  MISUNDERSTOOD 
(NOT  UNGUAGE  OR  COMPILER) 

0 

0.00 

CLERICAL  ERROR 

6 

3.12 

were  successfully  detected  through 
compiler  messages,  and  a  slightly 
smaller  number  were  successfully  de¬ 
tected  through  readings  and  walk¬ 
throughs.  These  same  activities  were 
used  to  isolate  the  source  of  the  error. 
Code  reading  was  more  successful  at 
isolating  than  detecting  the  source, 
and  the  opposite  is  true  of  compiler 
messages.  In  the  case  of  design  reading 
and  walkthroughs,  detection  of  the  er¬ 
rors  and  isolation  of  their  sources 
usually  occurred  simultaneously.  This 
information  indicates  that  careful 
design  and  code  reading  and  walk¬ 
throughs  should  be  stressed  and  that 
language  processors  should  be  used  as 
much  as  possible  to  detect  errors. 
However,  results  of  other  activities, 
such  as  test  Tuns,  would  surely  have 
appeared  here  if  more  testing  had  been 
done. 

As  with  the  changes,  most  of  the  er¬ 
rors  were  trivial.  More  than  80  percent 
took  12  minutes  at  most  to  isolate. 
Only  seven  errors  took  an  hour  or 
more  to  either  isolate  or  to  correct. 
One  error — a  design  incorrect  error 
that  involved  renaming  a  file— took  an 
hour  to  isolate  but  only  six  minutes  to 
correct.  Another,  classified  as  code  in¬ 
correct,  took  two  hours  to  isolate  but 
only  twenty  minutes  to  correct.  An 
undefined  part  of  a  string  was  passed 
as  an  argument  to  a  function.  Two  er¬ 
rors  involving  incorrect  design  each  re¬ 
quired  only  six  minutes  to  isolate  but 
more  than  an  hour  to  correct.  One  of 
these,  a  tasking  error  involving  a  syn¬ 


chronization  problem  between  two 
componenu,  took  5.2  hours.  Another, 
which  required  1 .5  hours,  was  a  logic 
error  involving  input/output.  The  re¬ 
maining  three  errors  took  an  hour  or 
more  to  isolate  and  another  hour  or 
more  to  correct.  One  required  the  in¬ 
sertion  of  error  checks  and  exception 
handlers  in  a  routine  conforming  to 
the  specifications;  this  took  one  hour 
to  isolate  and  one  hour  to  correct. 
Another  took  four  hours  to  isolate  and 
four  hours  to  correct;  it  was  an  in¬ 
put/output  syntax  error.  The  last  er¬ 
ror,  which  took  ofie  hour  to  isolate  and 
one  hour  to  correct,  was  a  require¬ 
ments  incorrect  error.  A  superfluous 
requirements  seaion  was  found  and 
eventually  deleted. 

Table  6  shows  the  number  of  com¬ 
ponents  changed  to  correct  each  error 
as  well  as  the  number  examined  while 
deciding  how  to  make  the  correction. 
(A  distinaion  is  made  between  errors 
in  general  and  those  which  caused 
compiler-deteaable  faults.  This  dis¬ 
tinction  will  be  described  in  more 
detail  in  the  next  section.)  Since  most 
of  the  errors  were  trivial  and  involved 
the  syntax  of  a  component,  most  of  the 
corrections  caused  only  one  compo¬ 
nent  to  be  changed  or  examined. 

Pos.sible  detection  by  tools.  Detec¬ 
tion  by  tools  is  one  method  of  classify¬ 
ing  faults.  The  classification  will  be  ex¬ 
panded  in  later  studies,  but  we  used  it 
here  to  separate  compiler-detectable 
from  noncompiler-deteciable  taults. 
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TabI*  6.  Number  of  components  involved  in  error  correction  (with  all  errors 
and  without  compiier  detectable  (auits)L 


Table  8b.  Number  of  reported  errors  in  module  (without  faults  detectable 
by  compiler). 


#  ERRORS 

#  MODULES  WITH  ERRORS 

TOTAL 

IN  MODULE 

ADA 

POL 

ADA 

PDL 

0 

LLSSSSSSSJJJJJJJJJJJJJJJJ 

LLUJJJJJJJJJJJ 

25 

15 

1 

LLSSSSSSJJJJJJB 

1 

15 

1 

2 

SSJJ 

LB 

4 

2 

3 

SS 

2 

0 

4 

S 

1 

0 

5 

S 

1 

0 

10 

S 

1 

0 

L;  Lead  programmer  S:  Senior  programmer  J:  Junior  programmer  B:  Librarian 


The  compiler -detectable  faults  are  fur-  able  by  a  processor  based  on  BNF 

ther  divided  into  those  related  to  the  should  be  eliminated  with  a  syntax- 

BNF  of  the  language  and  those  that  oriented  editor,  which  might  also 

might  require  more  information  than  eliminate  some  of  the  other  faults, 

the  BNF  contains.  Those  faults  delect  C'lUtipiler -detectable  faults  bas  e  often 


been  removed  by  the  time  data  collec¬ 
tion  begins  on  many  projeas.  There¬ 
fore,  the  data  from  which  those  faults 
have  been  removed  might  be  compar¬ 
able  to  early  development  data  in  later 
projects.  Table  7  lists  the  data  for  this 
projea  using  this  classification  scheme. 

Number  of  errors  per  module. 
Tables  8a  and  b  depict  how  many  er¬ 
rors  were  reponed  in  each  of  the  67 
modules  (Ada  and  non-expanded 
PDL).  Table  8a  shows  the  total  errors 
reported,  and  8b  itemizes  the  number 
of  errors  reported  in  which  the  fault 
was  not  detectable  by  a  compiler.  The 
letters  in  each  row  indicate  which  pro¬ 
grammer  wrote  the  module.  The  mod¬ 
ules  with  more  than  ten  errors  had  15, 
11,  20,  12,  and  27  reported  errors, 
respectively. 

Further  processing  after  project 
completion  showed  that  most  of  the 
non-expanded  PDL  modules  had 
compiler-detectable  faults  even 
though  no  errors  in  those  modules 
were  reponed.  This  Onding  indicates 
that  the  modules  w,ere  written,  ex¬ 
panded  into  code,  then  essentially  ig¬ 
nored.  The  data  in  Tables  8a  and  8b 
reinforce  this  observation.  The  senior 
programmer  seems  to  have  found 
more  of  the  less  obvious  errors  than 
other  programmers  since  his  modules 
have  more  reponed  errors  that  were 
not  compiler-detectable. 

Omission  or  commission.  Another 
type  of  error  classiHcation,  presented 
by  Basil!  and  Perricone,*  divides  er¬ 
rors  into  the  categories  of  omission 
and  commission.  Errors  of  omission 
leave  out  some  portion  of  code  while 
errors  of  commission  include  errone¬ 
ous  or  superfluous  code.  Table  9  pre¬ 
sents  the  data  for  this  project  as  well  as 
some  of  the  data  from  Basili  and  Per- 
ricone.  Note  that  the  percentages  for 
all  errors  from  this  study  and  in  new 
modules  from  the  earlier  study  are 
almost  the  same.  This  is  probably 
coincidence,  since  the  data  was  gath¬ 
ered  at  different  times  during  develop 
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Table  9.  Comparison  of  errors  of  omission  and  commission. 


1  .ERRORS  . 

1  INVOLVED 

RAW  ERRORS 

OMISSION j.;  :  .  COMMISSION 

.  i  \  PERCENTAGE 

OMISSION  .  .  1  COMMISSION 

lA- J 

This  study 

All  errors 

w/o  compiler  faults 

89 

23 

103 

21 

46 

52 

54 

48 

Basil!  &  Perricone 

All  errors 

79 

143 

36 

64 

New  module  errors 

52 

63 

45 

55 

Table  10.  Number  of  language,  problem,  and  clerical  errors. 


NUMBER  OF  ERRORS 

CATEGORY  ,  _ _ _ _ JMLERRORS _ W/0  COMPILER  FAULTS, 


Language 

Concept 

Semantics 

Syntax 

Problem 

Clerical 


160 

18 

8 

8 

44 

10 

108 

0 

26 

26 

6 

0 

ment,  and  our  data  are  incomplete. 
This  categorization  will  be  included  in 
some  of  the  following  tables.  Errors  of 
omission  generally  will  not  be  caught 
by  testing  with  a  structural  coverage 
criterion  and  may  be  overlooked  in 
code  reading. 

Language,  problem,  or  clerical.  We 

developed  yet  another  classification 
scheme  where  the  errors  are  identified 
as  language,  problem,  or  clerical. 
Language  errors  are  closely  related  to 
the  use  of  Ada  and  are  further  classi¬ 
fied  as  concept,  semantics,  or  syntax. 
A  syntax  error  involves  a  misunder¬ 
standing  or  misuse  of  the  syntax  of  a 
feature;  a  semantics  error  involves  a 
misunderstanding  of  the  meaning  of  a 
feature  in  that  language;  and  a  concept 
error  involves  a  misunderstanding  of  a 
feature’s  use.  The  problem  category 
results  from  a  misconception  of  the 
problem  domain  or  the  environment. 
Clerical  errors  include  those  due  to 
carelessness,  e.g.,  typographical  er¬ 
rors.  This  classification  is  somewhat 
subjective,  however,  since  the  project 
monitors  tried  to  determine  what  the 
programmer  was  thinking  when  the  er¬ 
ror  occurred. 

Of  the  192  error  description  forms 
examined,  160  (83  percent)  claimed 
that  the  use  of  Ada  contributed  to  the 
error.  As  shown  in  Table  10,  the  ma¬ 
jority  of  the  errors  were  language  er¬ 
rors,  and  67  percent  of  those  were  syn¬ 
tax  errors,  which  explains  why  so 
main  of  the  errors  took  so  little  time  to 
correct.  Almost  21  syntax  errors  oc¬ 


curred  per  thousand  lines  of  text  (any 
line  containing  part  of  an  Ada  state¬ 
ment)  and  almost  1 1  syntax  errors  per 
thousand  non-blank  lines. 

The  language-problem-clericaJ  clas¬ 
sification  can  be  used  in  conjunction 
with  the  document-type  classification 
as  seen  in  Table  11.  Not  surprisingly, 
most  errors  involving  requirements 
were  problem  errors,  and  most  of  the 
errors  involving  incorrect  design  or 
code  were  language-related  errors. 

Ada  language  features.  Several  Ada 
language  features  were  involved  in  er¬ 
rors.  Understanding  the  relationships 
between  errors  and  features  may  help 
prevent  the  errors.  Table  12  shows  the 
language  features  involved  in  errors, 
with  all  reported  errors  included.  In 
Table  13,  the  errors  that  caused  com¬ 
piler-detectable  faults  have  been 
removed. 

Low-level  syntax  (e.g.,  semicolon, 
parenthesis,  assignment),  loops,  decla¬ 
rations,  and  parameters  were  involved 
in  the  most  common  language  errors. 


Several  errors  also  involved  tasks, 
separate  compilation,  generics,  and 
procedures  and  functions.  As  pre¬ 
viously  stated,  most  of  the  errors  were 
compiler-detectable.  Only  eight  con¬ 
cept  errors,  which  involved  tasking, 
exceptions,  and  packages,  occurred. 
Of  the  44  semantics  errors,  nine  in¬ 
volved  parameters:  six,  generics;  five, 
compilation  units;  four,  declarations; 
and  three,  overloading.  However, 
many  of  those  could  be  detected  by  a 
compiler.  If  the  compiler-deteaable 
faults  are  removed,  only  10  semantics 
errors  remain,  and  four  of  those  in¬ 
volve  parameters. 

In  general,  few  serious  errors  were 
reported  in  this  project  because  little 
testing  was  done.  However,  the  data 
reported  suggests  the  types  of  errors  to 
expect  when  people  given  training 
similar  to  that  of  our  programmers 
learn  to  use  Ada.  Similar  errors  might 
be  made  when  learning  any  new  lan¬ 
guage,  however,  particularly  with  such 
a  large  number  of  new  features  and 
concepts. 


Table  11.  Type  of  document  vs.  language,  problem,  or  clerical  classification  and 
omission  or  commission  classification  (data  excluding  compiler  detectable 
faults  in  ( )). 


I  TYPEOF  DOCUMENT  '  '  NUMBER  OF  ERRORS 

.  AND  HOWlNyOLVED _ LANG, _ PROS _ CLEft  OMISSION  COMMISSION 


Requirements  incorrect 

0 

2(2) 

0 

1(1) 

1(1) 

Requirements  misinterpreted 

1(1) 

3(3) 

0 

3(3) 

1(1) 

Design  incorrect 

24(8) 

5(5) 

0 

14(6) 

15(7) 

Design  misinterpreted 

0 

0 

0 

0 

0 

Code  incorrect 

135(9) 

16(16) 

0 

70(13) 

81(12) 

External  environment 
misunderstood 

0 

0 

0 

0 

0 

Clerical  error 

0 

0 

6(0) 

1(0) 

5(0) 

Table  12.  Errors  categorized  by  Ada  language  feature. 


P  ADA  LANGUAGE  FEATURE 

... _ CON  . 

_SEM 

NUMBEROF  ERRORS 
.  SVN  OM  COM 

TOTAL. 

Semicolon 

0 

0 

17 

13 

4 

17 

f^renthesis 

0 

0 

12 

9 

3 

12 

Colon 

0 

0 

4 

2 

2 

4 

Assignment 

0 

0 

5 

1 

4 

5 

Strings 

0 

0 

4 

3 

1 

4 

Comment 

0 

0 

4 

2 

2 

4 

Identifler 

0 

2 

3 

0 

5 

5 

Loop 

0 

2 

8 

7 

3 

10 

Case 

0 

0 

1 

1 

0 

1 

If 

0 

0 

6 

2 

4 

6 

Begin/end 

0 

0 

4 

3 

1 

4 

.  Return 

0 

0 

1 

0 

1 

1 

Scoping 

0 

0 

2 

1 

1 

2 

Typing 

0 

2 

5 

0 

7 

7 

Aggregate 

0 

1 

0 

0 

1 

1 

Arrays 

0 

2 

2 

1 

3 

4 

Records 

0 

2 

2 

2 

2 

4 

Declarations 

0 

4 

8 

7 

5 

12 

Parameters 

0 

9 

5 

3 

11 

14 

Procedures  &  functions 

0 

2 

5 

2 

5 

7 

Access  type 

0 

1 

0 

0 

1 

1 

Tasking 

5 

0 

4 

4 

5 

9 

Exceptions 

2 

0 

1 

0 

3 

3 

Generics 

0 

6 

2 

3 

5 

8 

PKkages 

t 

0 

t 

2 

0 

2 

Compilation  units 

0 

5 

2 

5 

2 

7 

Attributes 

0 

1 

0 

0 

1 

1 

Pragmas 

0 

2 

0 

0 

2 

2 

Overloading 

0 

3 

0 

0 

3 

3 

B  Totals _ 

_  8  44 

_  10<^  _ 

71 

87 

Ada  use 

A  description  of  how  the  language  is 
used  is  the  fourth  goal.  Since  Ada  is 
complex.  It  was  thought  that  the  pro¬ 


grammers  might  begin  with  a  subset  of 
the  language.  .Ada  also  supports  a 
number  of  software  engineering  con¬ 
cepts  such  as  information  hiding  attd 
abstraction.  Assessing  Ada  use  might 


aid  in  the  evaluation  and  modification 
of  training  in  Ada  concepts  and  ap¬ 
plications.  In  addition,  tools  might  be 
developed  to  help  people  learn  to  use 
Ada’s  more  unusual  features. 

By  examining  its  simplest  features, 
we  discovered  that  except  for  the  goto 
and  code  statements  and  representa¬ 
tion  clauses,  the  progran.mers  used  all 
of  Ada’s  syntactic  features.  Tasking, 
generics,  packages,  exceptions,  and 
overloading  along  with  pragmas, 
aborts,  and  delays  were  used  nominal¬ 
ly.  However,  when  the  system  was 
designed,  the  programmers  did  not 
know  how  to  use  these  concepts  on  this 
application.  Therefore,  they  might 
have  been  uncomfortable  basing  their 
design  on  some  of  Ada’s  more  ad¬ 
vanced  features.  If  the  programmers 
had  more  examples  within  their  appli¬ 
cation  domain,  they  might  be  able  to 
take  advantage  of  these  features. 

We  also  looked  at  the  use  of  pack¬ 
ages  in  the  system  to  determine 
whether  concepts  such  as  data  encap¬ 
sulation  and  information  hiding  were 
used  effectively.  The  senior  and  junior 
programmers  defined  1 1  packages  for 
use  with  this  project;  however,  the  lead 
programmer  and  librarian  defined 
none.  Two  of  those  1 1  packages  served 
as  definition  common  blocks;  three 
were  libraries  of  functions;  four  de¬ 
fined  encapsulated  data  types  export¬ 
ing  private-type  definitions  and  opera¬ 
tions;  and  the  remaining  two  defined 
types  but  exported  the  representation 
of  the  type.  Of  the  four  packages 
which  defined  encapsulated  data 
types,  two  were  device  drivers  and  one 
was  a  mathematicil  function;  the  re¬ 
maining  package  definition  had  no 
corresponding  body.  This  indicates 
that  no  new  encapsulated  data  types 
were  defined.  The  programmers  used 
packages  for  types  they  had  used  in 
other  languages.  While  globally  visi¬ 
ble,  many  of  these  packages  were  not 
needed  globally,  indicating  that  the 
programmers  did  not  understand  the 
concept  of  information  hiding,  fiaii- 
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non  describes  the  use  of  packages  in 
greater  detail.^ 

Most  programmers  are  not  ac¬ 
customed  to  high-level  language  sup¬ 
port  for  their  concurrency  needs. 
Familiarity  with  concurrency  in 
another  language  would  be  of  little 
benefit,  however,  as  Ada  uses  an 
unusual  model,  rendezvous  and  tasks, 
for  concurrency.  We  wanted  to  know 
how  tasks  were  used  in  this  system. 
Although  the  system  was  designed 
with  communicating  tasks,  they  were 
at  a  high  level  and  had  little  com¬ 
munication.  Ten  tasks  were  defined: 
one  by  the  lead  programmer,  four  by 
the  senior  programmer,  and  five  by  the 
junior  programmer.  Except  for  two 
cases,  each  task  had  only  one  or  two 
entries,  and  since  the  system  was  not 
tested,  it  is  difficult  to  know  whether 
this  use  was  appropriate  or,  indeed, 
it  worked.  Only  further  experience 
with  tasks  would  determine  their  use. 
Training  in  tasks,  like  packages, 
should  probably  include  examples 
from  the  appropriate  application 
domain. 

We  also  sought  an  understanding  of 
the  exceptions  used.  However,  it  re¬ 
mains  unclear  when  exceptions  should 
be  handled  in  the  module  raising  them 
and  when  they  should  be  propagated. 
Nevertheless,  the  programmers  tried 
to  use  exceptions,  if  only  for  passing 
back  error  codes.  Twenty-one  of  the 
non-package  modules  had  exception 
handlers,  and  exceptions  were  raised 
explicitly  in  17  modules.  Without 
knowing  whether  the  system  runs,  it  is 
difficult  to  ascertain  whether  this  use 
of  exceptions  is  sufficient  or  ap¬ 
propriate. 

The  results  of  this  portion  of  the 
study  are  mixed.  While  the  program¬ 
mers  used  many  features  of  the  lan¬ 
guage,  it  is  difficult  to  determine 
whether  that  use  was  nominal  or  ap¬ 
propriate.  Furthermore,  we  know  little 
about  how  Ada  should  be  used.  Most 
examples  in  the  literature  arc  too  small 
to  compare  \sith  this  project.  How- 


Table  13.  Errors  categorized  by  Ada  language  feature  (without  compiler 
detectable  faults). 


ADA  LANGUAGE 
FEATURE 

CONCEPT 

NUMBER  OF  ERRORS 
SEMANTICS  OMISSION 

COMMISSION 

TOTAL  1 

Loop 

0 

2 

2 

0 

2 

Arrays 

0 

1 

0 

1 

1 

Parameters 
Procedures  & 

0 

4 

0 

4 

4 

functions 

0 

1 

1 

0 

1 

Tasking 

5 

0 

2 

3 

5 

Exceptions 

2 

0 

0 

2 

2 

Ganerics 

0 

2 

2 

0 

2 

Packages 

1 

0 

1 

0 

1 

Totals 

8 

10 

8 

10 

18 

ever,  no  discernible  subset  was  de¬ 
fined.  Other  than  the  code  statements 
and  representation  clauses,  which  were 
not  needed  for  this  application,  most 
of  the  language  was  used.  Therefore, 
use  of  a  “subset  compiler”  would  not 
have  been  appropriate  and  might  have 
limited  theproip-ammers’  design  of  the 
system. 

Programmer  differences 

The  fifth  goal  discussed  includes  a 
description  and  evaluation  of  the  dif¬ 
ferences  between  programmers  and 
their  use  of  Ada.  Programmers  with 
varied  backgrounds  might  use  Ada 
differently  and  might  make  different 
types  of  errors.  If  these  differences  are 
significant,  they  might  suggest  dif¬ 
ferences  to  be  seen  in  other  environ¬ 
ments.  They  might  also  suggest  how  to 
tailor  training  to  meet  the  needs  of  pro¬ 
grammers  with  varied  backgrounds. 

We  found  that  the  programmers 
used  most  of  the  language  in  basically 
the  same  way.  The  librarian  wrote  so 
little  code  that  drawing  any  conclu¬ 
sions  about  that  code  or  programmer 
would  be  presumptuous.  Other  than 
their  definition  and  use  of  packages, 
(he  other  programmers’  code  is  basi¬ 
cally  indistinguishable.  Either  we  do 
not  have  the  appropriate  techniques 


for  examining  their  code  or  they 
worked  so  closely  together  that  their 
individual  differences  are  hidden. 

Productivity  is  one  area  where  some 
differences  between  programmers  sur¬ 
faced.  While  the  rest  of  the  program¬ 
ming  team  produced  7.3  lines  of  code 
per  hour  spent  in  design  and  code  de¬ 
velopment,  the  senior  programmer 
produced  16.5  lines  per  hour.  By  all 
reasonable  measures  of  productivity, 
the  senior  programmer  was  most  pro¬ 
ductive.  The  junior  programmer  was 
somewhat  more  productive  than  the 
lead  programmer  and  the  librarian. 
The  fact  that  the  junior  and  senior  pro¬ 
grammers  wrote  the  most  code  and 
became  most  familiar  with  Ada  may 
explain  the  disparities  in  performance. 

The  only  marked  difference  among 
programmers’  errors  was  that  the 
junior  programmer  made  the  most 
language,  and  particularly  syntax,  er¬ 
rors.  However,  he  performed  the  most 
code  testing  and  therefore  had  the 
greatest  opportunity  to  discover  er¬ 
rors.  He  also  had  the  most  extensive 
background  in  software  engineering 
methodology,  which  seemed  to  help 
him  understand  how  to  use  Ada  and 
offset  his  lack  of  experience  in  the  ap¬ 
plication  area. 

Overall,  the  programmers  seemed 
to  write  code  using  the  features  of  the 
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language  they  thought  they  knew  best. 
The  senior  and  junior  programmers, 
who  had  varied  language  experience, 
used  the  more  Ada-like  features  such 
as  packages,  but  the  lead  programmer 
also  used  tasking  and  generics.  The  li¬ 
brarian,  with  little  language  experi¬ 
ence,  used  a  simple  subset  of  Ada.  He 
only  wrote  two  modules,  which  re¬ 
quired  only  a  subset  of  Ada.  None  of 
the  programme!  s  made  errors  remark¬ 
ably  different  from  those  of  the  others, 
although  further  testing  might  have 
shown  otherwise.  Productivity  ap¬ 
pears  to  be  the  only  aspect  of  this  proj¬ 
ect  that  could  be  used  to  differentiate 
programmers. 

Although  the  project  ended  before 
development  was  complete,  the 
results  indicate  what  might  happen  in 
early  stages  of  development  in  other 
projeas.  A  number  of  results  from  this 
project  might  prevent  others  from 
making  costly  management  mistakes. 

Above  all,  it  should  be  noted  that 
learning  Ada  takes  time,  a  factor  that 
will  influence  any  estimate  of  effort 
for  early  projeas  using  Ada.  Training 
will  probably  have  to  continue  as  team 
members  learn  the  finer  points  of  the 
language. 

Ada  is  more  than  syntax  and  simple 
examples.  The  underlying  software  en¬ 
gineering  concepts  must  be  taught  in 
conjunction  with  the  support  Ada  pro¬ 
vides  for  those  concepts.  Examples 
from  the  relevant  problem  domain  will 
help  students  fit  Ada  into  their  en¬ 
vironment.  Since  most  programmers 
are  not  familiar  with  the  mahodol- 
ogies  developed  in  the  70’s,  which  Ada 
supports,  training  in  software  engi¬ 
neering  methodology  and  its  use  in  the 
environment  of  a  particular  applica¬ 
tion  is  an  absolute  necessity. 

How  Ada  should  be  used  remains 
unclear.  Ideally,  our  understanding  of 
the  software  engineering  concepts  that 
Ada  supports  would  simplify  its  use. 
However,  many  people  learn  by  exam¬ 
ple,  and  good  examples  are  lacking. 


We  neither  know  how  nor  when  to  use 
exceptions,  tasks,  and  generics,  and 
can  only  gain  this  knowledge  by  study¬ 
ing  various  alternatives  and  showing 
how  they  wotk  with  examples  from 
various  environments.  In  this  respect, 
the  projea  has  raised  more  questions 
than  it  has  answered. 

Design  alternatives  must  be  investi¬ 
gated.  The  design  for  this  project  was 
funaional  and  more  like  than  unlike 
the  earlier  Fortran  design.  A  group  at 
General  Electric  developed  an  objea- 
oriented  design  for  the  same  projea,’ 
and  it  is  not  clear  which  design,  if 
either,  is  most  appropriate.  Just  as  a 
combination  of  top-down  and  bot- 
tom-up  development  is  appropriate  to 
many  applications,  a  combination  of 
funaional  and  objea-oriented  design 
might  well  be  most  appropriate.  Only 
by  determining  which  design  type,  or 
combination  of  types,  is  best  suited  to 
the  particular  application  can  we  teach 
people  which  design  approach  to  use. 
Without  such  training,  programmers 
must  rely  on  their  experience  with 
other  languages  and  will  probably  pro¬ 
duce  functional  designs. 

Proper  tool  support  is  mandatory. 
This  project  was  undertaken  without  a 
production-quality  validated  compil¬ 
er— a  necessary  tool.  Likewise,  a  lan¬ 
guage-oriented  editor,  capable  of 
eliminating  60  percent  of  the  observed 
errors,  would  have  been  desirable. 
Such  an  editor  would  have  freed  the 
programmers  to  concentrate  on  the 
logic  errors  that  undoubtedly  remain 
in  the  design  and  code.  Such  an  editor 
would  have  dramatically  reduced  the 
error  rate.  Other  useful  tools  for  this 
project  would  be  data  dictionaries,  call 
structure  and  compilation  dependency 
tools,  cross  references,  and  other 
means  of  obtaining  multiple  vievss  of 
the  system.  A  PDL  priKessor  with  in¬ 
terface  check.s,  definition  and  uve  rela¬ 
tion  lists,  and  various  metrics  would 
also  have  aided  in  the  carls  >i,iees  of 
development. 

Some  methods'loev  nu:>i  be  fol¬ 


lowed  for  a  project  to  be  successful, 
and  programmers  must  understand 
the  methodology'  and  tools  before  the 
project  begins.  In  this  case,  the  lack  of 
useful  tools  proved  troublesome.  In 
addition,  the  PDL  was  loosely  defined 
until  after  design  began.  Effective 
design  reading  might  have  caught 
many  errors.  If  we  had  tested  this  proj¬ 
ea  after  a  compiler  became  available, 
a  test  plan  created  after  the  require¬ 
ments  were  completed  would  have 
been  necessary.  However,  that  aspect 
of  the  methodology  was  deemed  unim¬ 
portant.  Language  is  only  one  aspect 
of  the  environment  and  methodology. 
It  cannot  save  a  project  in  which  the 
rest  of  the  methodology  is  ignored. 

We  believe  this  project  is  atypical 
since  it  was  not  fmished  and  no  compil¬ 
er  was  available.  However,  it  is  typical 
in  that  training  consumed  an  enormous 
amount  of  effort,  and  the  programmers 
were  not  familiar  with  the  underlying 
software  engineering  concepts  of  Ada. 
In  this  respect,  it  resembled  the  begin¬ 
ning  of  many  projects.  Also  of  note, 
the  learning  curve'in  methodology  is 
quite  large.  As  we  study  more  projects 
that  use  Ada,  we  will  learn  how  to  both 
teach  and  use  it  and  discover  how  to  re¬ 
duce  mistakes.  In  the  meantime,  we 
know  that  using  Ada  will  be  difficult  at 
first,  but  in  time  its  use  will  make  us 
more  effective  in  applying  existing 
software  engineering  techniques  to 
ease  the  programming  process  and 
thereby  increase  the  quality  of  the 
produa.  .1] 
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